{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With the functional API, not only can you build models with multiple inputs and multiple outputs, but you can also implement networks with a complex internal topology.\n",
    "\n",
    "acyclic mean = dont make cycle, are running left to right."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q. What is meant by directed acyclic graphs of layers ?\n",
    "\n",
    "Till now we saw in functional API that the layer output become the input of next layer. But now the output can be use not just for the next layer but also for any other forward layears as many time as we wish, but It cann't become the input of its parent layer (previous layer)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Several common neural-network components are implemented as graphs. Two\n",
    "notable ones are:\n",
    "\n",
    "1. Inception modules\n",
    "\n",
    "2. residual connections. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inception modules:\n",
    "Inception3 is a popular type of network architecture for convolutional neural networks;\n",
    "\n",
    "In which following the above principle they have created branches than concatenated them at the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Every branch has the same stride value (2),\n",
    "# which is necessary to keep all branch outputs the same size so you can concatenate them.\n",
    "\n",
    "from keras import layers\n",
    "\n",
    "branch_a = layers.Conv2D(128, 1,activation='relu', strides=2)(x)\n",
    "\n",
    "branch_b = layers.Conv2D(128, 1, activation='relu')(x)\n",
    "branch_b = layers.Conv2D(128, 3, activation='relu', strides=2)(branch_b)\n",
    "\n",
    "branch_c = layers.AveragePooling2D(3, strides=2)(x)\n",
    "branch_c = layers.Conv2D(128, 3, activation='relu')(branch_c)\n",
    "\n",
    "branch_d = layers.Conv2D(128, 1, activation='relu')(x)\n",
    "branch_d = layers.Conv2D(128, 3, activation='relu')(branch_d)\n",
    "branch_d = layers.Conv2D(128, 3, activation='relu', strides=2)(branch_d)\n",
    "\n",
    "output = layers.concatenate([branch_a, branch_b, branch_c, branch_d], axis=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the full Inception V3 architecture is available in Keras as \n",
    "\n",
    "keras.applications.inception_v3.\n",
    "\n",
    "InceptionV3, including weights pretrained on the ImageNet dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Residual connections\n",
    "Residual connections are a common graph-like network component found in many network architectures, including \n",
    "### Xception Model\n",
    "\n",
    "They tackle two common problems that plague any large-scale deep-learning model:\n",
    "vanishing gradients and representational bottlenecks.\n",
    "\n",
    "vanishing gradients = weights ke value bht ziada ye bht kam hojai, to gradiant vanishing problem hojate hai.\n",
    "\n",
    "bottlenecks = large layer ke bt, chote layer lagade, aur phr wapis bare layer lagade to, chote layer ke kam representaion (learning) ziada nahe ho pate to bottleneck problem hojata hai."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = ...  # input\n",
    "y = layers.Conv2D(128, 3, activation='relu', padding='same')(x)\n",
    "y = layers.Conv2D(128, 3, activation='relu', padding='same')(y)\n",
    "y = layers.Conv2D(128, 3, activation='relu', padding='same')(y)\n",
    "\n",
    "y = layers.add([y, x]) # wapis se input dedia take features barh jai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Residual connections mai Concatenate nahe sum kerte hai\n",
    "\n",
    "Contatenate me woh 2 values hote the\n",
    "\n",
    "but yaha sum kerke woh 1 value bnjate hai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import layers\n",
    "x = ...\n",
    "y = layers.Conv2D(128, 3, activation='relu', padding='same')(x)\n",
    "y = layers.Conv2D(128, 3, activation='relu', padding='same')(y)\n",
    "y = layers.MaxPooling2D(2, strides=2)(y)\n",
    "residual = layers.Conv2D(128, 1, strides=2, padding='same')(x)\n",
    "\n",
    "y = layers.add([y, residual])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q. What is the difference in Inception modules and Residual connections( Xception Model ) ?\n",
    "\n",
    "#### Answer: \n",
    "\n",
    "Residual connections mai woh sirf first input bar bar further layers ko dete hai, aur yaha layer add ker rahe hai.\n",
    "\n",
    "Japke Inception modules mai woh koi se bhe layer agay reuse ker rahe the, aur waha layers concatenate ker rahe the."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
